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Abstract 


Circular economy represents a fundamental alternative to the currently predominat- 
ing linear economy model, while Industry 4.0 is a technological enabler to bring pro- 
cess innovation in the industrial domain. New economic models are needed in order 
to reduce material inputs and waste generation leveraging on ecodesign, recycling and 
reusing of products, new business models, and new technologies. Internet-of-Things and 
artificial intelligence can support the circular economy paradigm, through the develop- 
ment of a marketplace for connecting buyers and sellers of manufacturing services, raw 
materials and products toward building global supply chains. The core component of 
this marketplace is a novel, agent-based, brokering module that will apply both syntactic 
and semantic matching in terms of manufacturing capabilities, in order to find the best 
possible supplier to fulfill a request for a service, raw materials or products involved in 
the supply chain. 


Keywords: circular economy, industrial symbiosis, Internet-of-Things, agent-based 
marketplace, artificial intelligence, waste management, trading 


1. Introduction 


Circular economy represents a fundamental alternative to the currently predominating linear 
economic model, based on the take-make-consume-dispose paradigm. The linear model has 
conducted to economic growth and welfare but ran its course. New economic models are 
needed in order to reduce material inputs and waste generation leveraging on ecodesign, 
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recycling and reusing of products, new business models, and new technologies. Products 
and production systems need to be designed to be “circular”: materials need to be efficiently 
processed and waste needs to be arranged and recycled. 


Looking from a technological perspective, the circular economy generates needs in the field 
of manufacturing, processing, networking, identification, and recycling of materials and 
products. As Banyai discussed [1], the success of the operation of these functions is based on 
the optimal design and control. Most of the supply chain models include only the traditional 
procurement-production-distribution sub-processes used for converting raw materials to 
final products and deliver them to the wholesalers, retailers or directly to end users. In this 
case study is showed a complex green supply chain model in which sophisticated operation 
research heuristics has to be used to find the optimal solution in order to minimize the costs. 


The main needs generated by circular economy are as follows: 


e Advanced collection, sorting, and recycling technologies— apps, sensors, robots, etc. 
e Efficient materials processing technologies— machine learning and artificial intelligence. 


e Production technologies that support design for circularity—3D printing, disassembly, 
and repairability. 


e Interactive platforms for enhanced connectivity —apps, websites, databases, and Internet- 
of-Things. 


Industry 4.0 is fundamental to move from linear to circular economy model. Industry 4.0 
refers to the introduction of new technologies, such as the Internet-of-Things, 3D printing, 
artificial intelligence, cloud computing, machine-to-machine communication, etc., in the tra- 
ditional industrial domain in order to support the improvement of the efficiency and automa- 
tion of the processes happening in the industry. 


The introduction of circular economy generates new technological and nontechnological 
needs. The change in ownership and material management concepts, both at a consumer and 
at business level, generates a need for upscaling and acceleration of business concepts such as: 
products as a service, sharing platforms, peer-to-peer interactions, and industrial symbiosis 
[2, 3]. Industrial symbiosis approach focuses on the hidden value of waste resources within an 
industrial network which can be exploited through the cooperation [4]. 


Wasted resources are unused or reusable energy, water, materials, and residues of produc- 
tion processes. Industrial Symbiosis can help improve the overall efficiency of the industrial 
system: companies within an industrial symbiosis network establish beneficial business rela- 
tionships among them based on these exchanges of resources. However, these relationships 
are not always easy to establish due to the difficulty in creating the network or lack of a 
communication platform for the network [5]. 


The vision of the circular economy is well suited to the concept expressed by the manifesto of 
the Physical Internet (PI) Initiative [6], a recent concept of breakthrough innovation aiming to 
improve by an order of magnitude; the economical, environmental, and societal efficiency and 
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sustainability of the way physical objects are moved, deployed, realized, supplied, designed, 
and used. It attempts to achieve this scope by applying concepts from Internet data transfer to 
real-world shipping processes [7, 8]. 


Previous research has already proven that new inventory models enabled by and applied 
in PI could help reduce inventory levels, thanks to its high flexibility [9]. The vision of the 
physical Internet involves encapsulating goods in smart, ecofriendly, and modular containers 
ranging from the size of a maritime container to the size of a small box. That new inventory 
models enabled by and applied to PI could help reduce inventory levels, thanks to its high 
flexibility. These modular containers will be continuously monitored and routed, exploiting 
their digital interconnection through the Internet-of-Things. 


Circular economy concepts, and more specifically industrial symbiosis, are vehiculating lot of 
research in the Internet-of-Things (loT) domain. Indeed, waste characterization and identifi- 
cation can be considered as the most challenging steps in waste management, that is, the ones 
that kick off the subsequent valorization and exploitation. Once characterized and identified 
the waste, the loT-domain literature proposes a huge number of infrastructures and platforms 
(e.g., see [10-12]), which helps to monitor features of interest, in real time or on historical base, 
communicating information only to subjects who are subscribed to specific topics of interest, 
just when they are available. 


The aim of this chapter is to present how Internet-of-Things and artificial intelligence can 
support the circular economy paradigm, through the development of a marketplace for con- 
necting buyers and sellers of manufacturing services, raw materials, and products toward 
building global supply chains. The core component of this marketplace is a novel, agent- 
based, brokering module that will apply both syntactic and semantic matching in terms of 
manufacturing capabilities, in order to find the best possible supplier to fulfill a request for a 
service, raw materials or products involved in the supply chain. 


Section 2 provides the analysis of the background and state of the art of the multi-agent 
systems. 


Section 3 of this chapter is dedicated to the description of a typical industrial symbiosis case 
study that the proposed solution intends to address. 


Section 4 introduces the solution proposed by this work to support the circular economy, 
which integrates traditional IoT platforms with an agent-based approach to support a bidding 
system through the implementation of a marketplace. 


Section 5 specifies the type of agents implemented in the marketplace, and finally Section 6 
concludes the chapter addressing the future works to extend the platform. 


2. Background and state of the art 


Multi-agent systems (MAS) [13] have been widely studied in the literature, and some existing 
options have been evaluated before deciding to implement the marketplace. A multi-agent 
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system is a computerized system composed of multiple interacting intelligent agents. Agent 
platforms are typically designed with a container paradigm, where all agents live in a well- 
defined agent container. Among the many platforms available on the Internet, a few subsets 
of candidates that might cover a good subset of composition features and requirements have 
been selected. 


JIAC (Java-based Intelligent Agent Componentware) [14] is a Java-based agent architecture 
and framework that allows an easy development as well as the operation of large-scale, dis- 
tributed applications, and services. The framework supports the design, implementation, 
and deployment of software agent systems. Then, the entire software development process 
is supported by JIAC. It also allows the possibility of reusing applications and services, also 
by modifying them during runtime. The strength points of JIAC are distribution, scalability, 
adaptability, and autonomy. In fact, JIAC V applications can be developed using extensive 
functionality that is provided in a library. This library has already-prepared services, compo- 
nents, and agents which can be integrated into an application. For example, individual agents 
are based on a component architecture which already provides the basic functionality for 
communication and process management. However, application-specific functionality can be 
provided by the developer and be interactively integrated. 


SARL [15] is a general-purpose agent-oriented language. SARL focuses on providing the 
fundamental abstractions for dealing with interaction, concurrency, distribution, reactivity, 
decentralization, autonomy, and dynamic reconfiguration. These are high-level features con- 
sidered so far as the major requirements for a good and practical implementation of modern 
complex software applications. SARL approach remains as generic as possible and highly 
extensible to easily integrate new concepts and features comparing to the variety of exist- 
ing approaches and meta-models in the field of agent-oriented engineering and multi-agent 
systems. The language is platform and architecture independent. 


JADE (Java Agent DEvelopment Framework) [16] is a software framework that eases the 
implementation of multi-agent systems through a middle-ware complies with the FIPA [17] 
specifications and a set of graphical tools that help the debugging and deployment process. 
Another helpful characteristic concerning JADE-based system is that it can also be distributed 
across machines with different OS. In a distributed scenario, the configuration of each system 
deployed can be controlled remotely thought a GUI. The configuration can be even changed 
at run time by moving agents from one machine to another, if required. Finally, JADE is com- 
pletely implemented in Java language, and the minimal system requirement is the version 5 
of JAVA. 


SPADE (Smart Python multi-Agent Development Environment) [18] is a multiagent and orga- 
nizations platform based on the XMPP/Jabber technology and written in the Python language. 
This environment offers features and facilities that simplifies the construction of MAS, exam- 
ples are: the concepts of users (agents) and servers (platforms); an existing communication 
channel and an extensible communication protocol based on XML, just like FIPA-ACL [19]. 
Many other exist, but SPADE has been the first agent platform based on XMPP technology. 


In Table 1, a comparative summary of the candidate agent platforms is reported, where 
respective pros and cons are addressed. 
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Platform Pros Cons License 
JIAC Exploits Apache ActiveMQ as transport declarative Not FIPA compliant Apache v2.0 
agent definition through spring 

SARL Exploits ZeroMQ as transport Not FIPA compliant Apache v2.0 
JADE FIPA compliant Supports only RMI transport. LGPL 

Few extensions for JMS/XMPP 

transports 
SPADE FIPA compliant Uses a custom XMPP server LGPL 


Table 1. Candidate agent platforms. 


As it is possible to notice from the presented analysis, none of the platforms offers both a dis- 
tributed approach and a general purpose implementation for the communication back end. 


The platform proposed in this work, therefore, represents a unique example of a marketplace 
based on semi-automatic multi-agent collaboration. 


3. Case study 


Internet-of-Things can support the circular economy in many ways. A typical case study 
consists on the early and automated detection of scrap material levels and its management, 
also identifying a network of companies which take waste materials as raw material for their 
production process. 


Whenever a desired level (e.g., 50, 75 or 90%) is detected by the scrap material storage area 
sensors, it will be annotated, reported, and recorded through IoT platform automatically. 
Getting accurate information about the scrap material storage area levels during the day will 
now be possible and the company producing the scrap material will stop occupying the staff 
time to manually detect scrap material levels at each storage area, with the system taking care 
of such procedure in each factory. Real time data will help optimize scrap material manage- 
ment procedures. Scrap material recycling companies will be either automatically contacted 
by the platform or they will receive continuous information on the current scrap material 
level. In response to such an interaction, they will be ready to put their offers to the system, 
once anew bidding process will be started by the company producing the scrap material. Such 
process can take place once the scrap material specification and the bidding specifications 
have been identified, with the company inserting the bid details in the platform. The system 
registers the new bid request and contacts the waste management companies for getting bids 
on scrap material. Companies contacted by the system typically belong to a selected subset 
of contractors. However, according to the company’s policies, in some cases, the bid could be 
open to new contractors, for example, to find better offers fostering new collaborations. When 
the system contacts contractors for bidding, a deadline is provided on when the last bid will 
be accepted. Once the deadline is reached, the platform system will select the best (or n-best) 
offer according to the bidding process criteria (entered when the process started) and present 
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Marketplace platform Waste management companies 


Figure 1. Scrap material collection and bidding process. 


it to the relevant manager (this automation will reduce the time required for processing bids). 
Upon human approval, bidders will be automatically informed about the outcomes of their 
bids. The company will then arrange a pick-up date and time in agreement with the selected 
scrap material recycling company; such an arrangement will be automatized to the greatest 
extent (see Figure 1). 


The criteria to award the contract will be price, company reputation, and the data agreement 
for the collection of the material. The literature presents several well-known algorithms that 
can be leveraged for automatic negotiations, such as contract net protocol [20], while many 
protocols are nowadays considered de facto standards for supporting secure peer-to-peer 
communication in IoT (such as, XMPP, MOTT, CoAP, and AMQP). 


4. The bidding platform a.k.a. marketplace 


This section introduces the solution proposed to support the circular economy in the use 
case presented in Section 3. The proposed platform integrates traditional loT platforms with 
an agent-based approach to support a bidding system through the implementation of a 
marketplace. The marketplace is a fully distributed multi-agent system designed to support 
Industry 4.0 exchanges between involved stakeholders. It is particularly aimed to supporting 
automatic supply chain formation and negotiation of goods/data exchanges. The marketplace 
exploits a microservice architecture, based on Docker, and relies upon a scalable messag- 
ing infrastructure. Trust and security are granted in every negotiation step undertaken by 
automated agents on behalf of involved stakeholders. 
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Figure 2. Marketplace components overview. 


The marketplace includes the following elements: (a) a marketplace management portal; (b) a 
marketplace core based on Docker; and (c) a set of “default” agent implementations ready to 
be adopted by interested stakeholders. 


The main building blocks of the marketplace are displayed in Figure 2. 


4.1. Agent management system 


According to FIPA specifications [21], an Agent Management System (AMS) is a mandatory 
component of every agent platform and only one AMS should exist in every platform. It 
offers the White Pages service to other agents on the platform by maintaining a directory 
of the agent identifiers currently active on the platform. Prior to any operation, every agent 
should register to the AMS to get a valid agent identifier, which will be unique within the 
agent platform. 


4.2. White Pages service 


A White Pages service is a mandatory component of any MAS system. It is required to locate 
and name agents on the system, making it possible for one agent to connect with one another. 
In the current implementation of the Agent Management System, the agent identifiers are 
stored in a MySQL database. MySQL has been chosen because it offers relevant features for 
the project such as on-demand scalability, high availability, and reliability. Other agent plat- 
forms, like SPADE, use MySQL as well for the database underlying the White Pages service. 


Agents are stored according to the schema depicted in (Table 2): 
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Agent_id Agent_owner Agent_role 
The unique identifier for the agent. It is The name of the company The role of the agent on the marketplace 
the primary key for the table, having the owning the agent. can be either ‘Requester’ or ‘Supplier’. 


constraint of being unique. 


Table 2. Table schema. 


Since the directory service is offered by the AMS, the latter is the only agent allowed to directly 
interact with the database. When the AMS is executed, it needs the following configurations: 


e IP address of the host running the MySQL instance. 
e Valid username and password for performing CRUD operations on the database. 


e Name of the database to operate on, since the same instance may run different databases. 


5. Agents in the marketplace 


There are two main categories of agents that can be defined a priori, depending on the kind 
of services provided: 


e Marketplace agents (MA) 
e Stakeholder agents (SA) 


The first category groups all the agents providing services that are crucial for the marketplace 
to operate. The second category, instead, groups agents developed and deployed by the mar- 
ketplace stakeholders to take part in chain formation rounds. 


From an implementation point of view, they are very similar and share a large set of features, 
especially the communication protocols used for the interaction with stakeholders and other 
agents. 


The communication protocols used by the MA for the interaction with the stakeholder on 
left side and with other agents on the marketplace on the right side are shown in Figure 3. 
Any agent can be controlled by the stakeholder via RESTful APIs which will be listed and 
explained in the following sections. Agents on the marketplace can exchange messages using 
Advanced Message Queuing Protocol (AMQP) as transport layer. 


Stakeholder agents are instead deployed at the stakeholder’s premises, and their purpose is to 
fulfill the stakeholder’s interests. Stakeholder agents can be further divided into two different 
categories: requester and supplier. In the following, the reference implementations for the two 
different kinds of stakeholder agents will be described. However, both kinds of agents share 
a set of features allowing them to respond in different ways to the same kind of solicitations 
(such as input from GUI, messages from other agents) according to the state of their current 
behavior (when the solicitation is received). 
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Agent 


Marketplace 


Agentservice /€———>| Agentinstance 


Stakeholder 


Figure 3. Simplified agents’ communication logic. 


Currently, agents support two different behaviors: 


1. Authentication behavior: This is triggered upon the each agent activation. It consists of all 
the procedures that are necessary for the agent to correctly operate on the marketplace. 


2. Negotiation behavior: This is the behavior each agent needs to support in order to be able 
to participate in marketplace negotiations. It is designed in a way that allows different 
negotiation protocols to be adopted. 


The authentication behavior is the first one engaged by any agent since: 


1. It takes care of getting the agent identifier from the AMS. 


2. It authenticates the agent with the messaging broker, allowing secure message exchange 
over SSL. 


Both these phases are mandatory for any agent to operate on the marketplace. After the 
authentication process, the negotiation behavior comes to play. It allows the agent to perform 
transactions and interactions with the other agents on the marketplace. Its behavior is con- 
strained to the negotiation protocol that is being used. 


Agents might or might not support a certain level of “intelligence” in their decisions regard- 
ing certain negotiation protocols. For example, an agent might have the capability of self- 
evaluation for the received offers, whereas another agent might not have such capability and, 
therefore, it will involve an external agent (providing such intelligence) during the decision 
process. An agent having such level of intelligence will be described as “smart,” while it will 
be described as “dumb” in the absence of that. 


5.1. Requester agent 


The requester agent is the agent used by a factory to request the execution of an existing 
supply chain or to initiate anew supply chain. Due to the dynamics of exchanges pursued in 
a factory-oriented marketplace, there is no actual distinction between the two processes, that 
is, for any supply need a new chain is formed, and a new execution of the chain is triggered. 
The requester agent may act according to several negotiation protocols, which can possibly 
be supported by only a subset of the agents active on a specific marketplace instance. In 
the proposed implementation, it exists a baseline protocol, which must be supported by 
any agent, called CONTRACT-NET. In such protocol, a requester agent plays the role of 
“Initiator.” 
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WAITING 
PROPOSALS 


Figure 4. Requester states. 


As described by Shoham [22], any agent can pass through a different set of states; the state 
of an agent consists of components such as beliefs, decisions, capabilities, and obligations. In 
the CONTRACT-NET, protocol requester agents go through different sets of states, perform- 
ing different actions upon the receipt of a message (either from the UI or from other agents) 
according to their current state. In Figure 4, the states for requester agent are shown with 
arrows indicating the allowed transitions between one state and another. 


5.1.1. Requester agent negotiation protocol 


One of our factory-oriented marketplace strengths is the tight connection existing between 
intra-factory and inter-factory components. This connection allows the agent on the market- 
place to operate in the optimum way to fulfill the stakeholder’s needs. 


Concerning the requester agent, such connection is the starting point of a new negotiation 
protocol: the IIMS notifies, through the learning agent (an agent available at the stakehold- 
er's premises, in charge of intra-inter factory communication), that a certain good/service is 
needed. Upon receipt of such notification, the requester agent can react accordingly, using the 
most suitable protocol to fulfill the request. A possible interaction is here described from the 
requester agent point of view: 


1. Marketplace system automatically notifies the stakeholder about the scrap metal bin fill 
level, through a notification from the learning agent (which receives the data from the 
sensors installed in the factory premises) to the requester agent. 


2. Requester agent has the logic for starting a new bidding process through default param- 
eters (set by the stakeholder). 


3. Requester agent starts the bidding process, looking for the list of the supplier agents cur- 
rently available on the marketplace that are capable in replying to such offer. 


4. Offers come to requester agent, which can either evaluate them locally (smart agent behav- 
ior) or send them out for evaluation toward an external agent. 
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5. Once the deadline for receiving proposals has expired, requester agent notifies the UI 
about the best one(s) that have been received. 


6. Stakeholder can now choose the preferred offer, select it, and notify the requester agent 
that can close the deal with the corresponding supplier agent. 


The same input that is needed for a bidding process to start, that is, the fill level, can be used 
by the requester agent to update the supplier agents interested in receiving such information 
in order to be ready when a new bidding process will take place. The latest described scenario 
is here described from the requester agent’s point of view: 


1. The fill level information is received from the learning agent. 


2. The requester agent forwards such information, in the marketplace, only to the supplier 
agent interested in receiving it. 


5.2. Supplier agent 


The supplier agent is the counterpart of the requester agent on marketplace. It is usually 
adopted by actual suppliers to respond to supply requests coming from other stakeholders in 
the marketplace. Factories transforming goods typically employ at least one requester agent 
to get prime goods and one supplier agent to sell intermediate products to other factories. 


In the CONTRACT-NET protocol, supplier agents go through different sets of states, per- 
forming different actions upon receipt of a message (either from the UI or from other agents) 
according to their current state. In Figure 5, the states for supplier agent are shown with 
arrows indicating the allowed transitions between one state and another. 


5.2.1. Supplier agent negotiation protocol 


A typical supplier agent is registered on the marketplace remaining “silent” while waiting for a 
new call for proposal, issued by a requester agent. Once this message arrives, the supplier agent: 


1. Evaluates the call for proposal message according to the policies set by the agent owner. 


2. If the proposal is compliant with such policies, an offer is sent from the supplier agent, 
otherwise it goes back to the “silent” (IDLE) state. 


3. Once the offer has been sent, the supplier agent waits for the requester’s decision. 


4. Inthe end, if the requester agent has accepted the proposal, a notification from the supplier 
agent is sent to the UI for notifying the stakeholder and waiting for his decision. 


5. The final message for acceptance is sent back to the requester agent in case the stakeholders 
accept the deal, otherwise a reject message will be sent. 


As already mentioned for the requester agents, the tight connection between intra-factory and 
inter-factory systems is a strength for the marketplace. For the supplier agent, this connection 
is exploited to dynamically adapt to the market needs. 
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Figure 5. Supplier states. 


For the supplier agent, this connection is here briefly described from the supplier agent point 
of view: 


1. The supplier agent subscribes, via AMOP, to the queues where the fill level notification 
related to a good of a certain company will be received. 


2. When such information arrives, the supplier agent can store it to exploit it for predictions 
of future fill levels. 


The same behavior can be used for predicting the prices of a certain good/service. The sup- 
plier agent can, in fact, store information about the past bidding process and apply deep 
learning techniques to have a prediction about the possible future values of interest, that is, 
what will be the selling price for a certain good at a certain point of time in the future. 


6. Conclusions and future works 


The proposed marketplace platform shows an approach that enables the paradigm switch 
from the linear economy model to the circular one. The circular economy paradigm is based 
upon reuse, reparation, refurbishment, and recycling of existing materials and products and, 
as shown in the presented case study, these activities can be facilitated through the proposed 
marketplace approach. Particularly, acompany can order the raw materials exploiting a cer- 
tain supply chain available on the marketplace, process them, and sell the residual production 
waste through another dedicated supply chain available on the marketplace. On the other 
hand, a recycling company can buy (or simply collect) the offered waste, properly process it, 
and sell it again as raw material. 


What is important to notice is that, even though existing collaboration models can be exploited 
on the marketplace, the proposed implementation has been designed to foster new collabora- 
tion models within and across the supply chain, making it easier for new actors to participate 
and collaborate with the existing ones. In order to increase the overall market competitivity 
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while trying to maximize the participant profits, new protocol interactions based on differ- 
ent kinds of auctions are being studied. For instance, Vickrey auction [23] is being investi- 
gated, since it has been proved to be efficient, under certain circumstances, in maximizing the 
expected utility of each bidder. 


One of the next important issues to be carefully addressed concerns the trust. Platforms like 
eBay imply the complete trust of strangers who take money before sending any wished goods. 
The reputation of sellers is built according to feedback provided by previous buyers and often 
is very hard for newcomers to sell items on the marketplace because they have to build up a 
good reputation. With this trust system is very easy to fake reviews in such a way to increase 
the reputation. Nowadays, Distributed Ledger Technologies (DLT), such as blockchain, are 
analyzed to be applied in many different application domains. Marketplaces can benefit from 
the power of DLT since it is built to create trust and transparency. In order to enhance the 
proposed marketplace approach, the authors are planning to leverage on DLT. DLT have 
received more and more attention in recent years as innovative methods to store and update 
data within and among organizations. Multiple copies of the ledger are held by different 
parties, with data added by consensus and without the need for a third party. This means 
that DLT are able to offer an immutable record, which means that data added to the ledger is 
in theory unchangeable, secure, and preserved for the life of the ledger, with the agreement 
of all participants. Additions to the ledger or changes to the governing structure are decided 
on a consensus basis by multiple participants. So, these systems provide a transparent and 
verifiable record of transactions. As a result, DLT can provide gains in trust among bidding 
participants. 
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